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(54) Multiple path routing 



(57) . A novel data structure in a router helps to com- 
pute viable next hops for forwarding a data packet from 
a router to its destination along multiple alternate loop- 
free paths, which are not necessarily of shortest dis- 
tance. Each ^viable next hop may also be specified with 
a degree of optimality, which enables a route to perform 
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QoS routing and fault-tolerant routing efficiently. The da- 
ta structure can be implemented as an add-on software 
to existing routing protocols and may be implemented 
in existing networl<;s which use shortest path protocols, 
even where less than all of the routers use the data 
structure and multiple path scheme described herein. 
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Description 

RELATED APPLICATIONS 

[0001] This application claims the benefit of provision- 
al U.S. Patent Application Serial No 50/056,027 filed 
September 2, 1997. 

FIELD OF THE INVENTION 

[0002] The present invention relates generally to rout- 
ing data packets to a destination node within a packet 
switching network. In particular the invention relate to a 
multiple path routing scheme using a novel data struc- 
ture. 

BACKGROUND OF THE INVENTION 

[0003] In packet switching networks such as the In- 
ternet, a packet from a source traverses intermediate 
network routers before reaching its destination. Before 
a packet arrives at a router, a routing protocol is used 
to determine the existence and status of all routers and 
links therebetween on the network This information is 
used by the router to determine an appropriate next hop 
for an arriving packet with a given destination. The next 
hop includes the next router along the path to the des- 
tination and the outgoing port of the present router link- 
ing the next router. A widely used Itnk-stale routing pro- 
tocol, is Open Shortest Path First ("OSPF") 
[0004] Giyen the current link stales in the network, (e. 
g. reduced bandwidth, increased bandwidth, discon- 
nected, reconnected) each router S Keeps track of a 
shortest path tree "SPT" from S to every other router in 
the network. Since the router retains only one shortest 
path from S for each destination router the first hop from 
S along this path will be used as the next hop to forward 
any packets with the same given destination 
[0005] Although a router using OSPF has global in- 
formation about the regional network topology, it deter- 
mines only the local route of a packet t e. the next hop. 
When each router computes such next-hop information 
based on an SPT, each packet is guaranteed to be for- 
warded along a loop-tree path to its destination. A loop- 
free path is a path that doesnl include the same router 
more than once. Multiple SPTs rooted at router 8 can 
co-exist in a network. In such a case there are multiple 
choices of next hops from S to forward a packet, each 
of which will lead to a different path ol itte same shortest 
distance. The packet can be forwarded to any of these 
next hops. 

[0006] Routing protocols allowing lor only a single 
shortest path suffer from several inefficiencies. For ex- 
ample, if immediately before dispatching a packet along 
its next hop, a router learns that a link along the path is 
down, the router must first recompute a new single 
shortest path and only then send the packet along the 
new next hop. 



[0007] Th single shortest path method also ignores 
network load considerations. Thus, even if a path has a 
lot of traffic the packet will be fonvarded along that path 
if it is. the shortest to its destination. Furthermore, it is 

5 known that finding an optimal path that satisfies certain 
QoS constraints is in general computationally difficult 
(NP-complete). Sirlce some QoS parameters that need 
to be optimized may depend on the network traffic load 
and thus can change dynamically over a short period of 

TO time, finding an optimal path can become infeasible 
even for a moderate-size network. Even for QoS param- 
eters that do not change frequently, determining the op- 
timal path from a router may require global knowledge 
of these parameters in the network, and a local change 

^5 in these parameters would therefore need to be dissem- 
inated to every node in the network. Hence in most cas- 
es only suboptimal paths can be computed. 

SUMMARY OF THE INVENTION 

20 

[0008] The present invention uses a multiple path al- 
gorithm (referred to herein as "MPA") to enable a router 
to fonward packets to multiple viable next hops that are 
not strictly constrained to be a part of a shortest path 

25 from the router to the packet's destination. The method 
of the present invention still guarantees, however, that 
all packets will be routed totheirdestination on loop-free 
paths. In general each path traversing a different next 
hop is guaranteed to be loop free if it satisfies the con- 

30 straint that the shortest distance from the implementing 
router to the destination decreases at each next hop. 
[0009] In a preferred embodiment of the invention, the 
MPA uses a novel data structure which stores informa- 
tion relating to each router (node) in the network that is 

35 a potential destination node. In partbular, the data struc- 
ture maintains at least the following attributes: the short- 
est distance from the implementing router to the desti- 
nation node; the cost of each link from the implementing 
'router to each potential next hop; and for each potential 

40 next hop, the shortest distance from the implementing 
router to the destination node along a path traversing 
that particular next hop. 

[0010] Using this data structure, multiple viable riext 
hops are computed by selecting next hops from the po- 
45 tential next hops whose shortest distance attribute tra- 
versing that next hop. minus the cost of the link from the 
implementing router to the next hop, is less than the 
shortest path from the implementing router to the desti- 
nation node. 

so [0011] The method and data structure of the present 
invention can be efficiently implemented as an add-on 
component to existing routing protocols such as OSPF. 
Each MPA implementing router uses the topology infor- 
matran xchanged by the router protocols to compute 

55 multiple paths b twe n a given source and a giv n des- 
tination. Moreover, conventional routers based on short- 
est path routing can co-exist in the same network with 
MPA implementing routers to guarantee loop-free rout- 
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ing for each packet, in general, an MPA implementing 
router can interoperate with any router whos routing 
protocol uses the loop-tree policy. 
[0012] By maintaining multiple viable next hops in a 
router for each destination, recovery from link failure is ^ 
much faster than heretofore possible, since the recovery 
mechanism using alternate paths can be implemented 
locally by the router, and there is almost no delay in- 
curred by the link failure. On the other hand, without 
MPA, after detecting a link failure, the router would need 
to broadcast throughout the network the link failure, then' 
recompute the shortest path tree to determine a new al- 
ternative path. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0013] 

FIG. 1 is a graphical depiction of the data structure 
used in accordance with the present invention. 
FIG. 2 is a graphical depiction of the data structure 
used in a conventional shortest path tree algorithm. 
FIG. 3 depicts a router architecture employing the 
MPA algorithm as used in accordance with the 
present invention. 

FIG. 4 is a representation of a network of routers 
including a source router S and a destination router 

D- 

DETAILED DESCRIPTION OF THE INVENTION 

[0014] The packet routing method of the present in- 
vention involves making available at each router, alter- 
native multiple next hops to forward a received packet. 
The only constraint in computing the multiple next hops 
is that the paths to the packet's destination resulting 
from these next hops do not return the packet to any 
router it previously hit. In other words, the paths of these 
altemative next hops should be loop free. As discussed 
above, the OSPF protocol well known in the art, guar- 
antees loop-free routing to each destination by con- 
straining each router to choose the next hop based on 
an SPT. However, a loop free path can be guaranteed 
ven if it does not represent the shortest distance to a 
destination. For example, any path to a destination that 
traverses a particular next hop from a router S is loop 
free if the distance to the destination continues to de- 
crease at each next hop. 

[001 5] I n one preferred embodiment of the present in- 
vention viable next hops from a router S, i.e., next hops 
which lie on a loop free path to a destination, can be 
guaranteed where the shortest distance from S to the 
destination traversing a given next hop, minus the cost 
of the link (i. ., its distance) from S to that next hop is 
less than the absolute shortest distance from S to the 
destination. For example, if router R is the next hop from 
router S along a shortest path from S to destination rout- 
r D, then the distance between R and D, i. d(R, D), 



must be less than the distance from S to D, i.e., d{S, D). 
Extrapolating R to any next rout r Rj, wrtiich can even 
be a router not on the absolute shortest path between 
S and D. a loop free path from S to D is guaranteed with 
a next hop to Rj if d(Rj, D) < d(S, D), 
[0016] Referring to FIG. 4 an example of a network of 
routers is shown, including source router S, destination 
router D and a plurality of routers R. Each link between 
the routers is numbered and it is assumed that each link 
has the same cost, or distance. In this example, it js clear 
that there are three shortest paths: (i) 1 -2-3; (ii) 7-9-10; 
and (lii) 4-14-3. 

[0017] However, in addition to these shortest paths, 
there are at least twenty four loop free paths. In partic- 
ular. 1-5-6- 14-3; 1-5-6-15-16-3; 1-5-6-15-12-13; 
1-5-6-8-9-10; 1-5-6-8-9-11-16-3; 1-5-6-8-9-11-12-13; 
4-5-2-3; 4-6-15-16-3; 4-6-15-12-13; 4-6-15-11-10; 
4-6-8-9-1 0; 4-6-8-9-11 -1 6-3; 4-6-8-9-11 -12-13; 
7-9-11-16-3; 7-9-11-12-13; 7-8-15-16-3; 7-8-15-11-10; 
7-8-1 5-12-1 3: 7-8-6-14-3; 7-8-6-14-16-12-1 3; 
7-8-6-14-16-11-10; 7-8-6-5-2-3; 7-8-6-5-2-16-11-10; 
7-8-6-5-2-16-12-13. A non-viable loop path, however; 
exists for example on path 7-8-6-5-2-1 6-15. . . since link 
15 will return the packet to a router previously visited 
between links 8 and 6. 

[0018] In the preferred embodiment of the invention, 
the MPA of the present invention makes use of the data 
structure of FIG. 1 to compute alternate viable next hops 
from a router whose shortest distances to the destina- 
tion are less than the shortest distance from the router 
to the destination. 

[0019] During execution, a conventional SPT algo- 
rithm usually keeps track of certain state infomnation 
about each node in the network. See E. Dijkstra, "A Note 
to Two Problems in Connection with Graphs" Numerical 
Mathematics, vol. 1 p. 115-126 (1959); R. Bellman, "On 
a Routing Problem" Quarterly of Applied Mathematics, 
vol. 16, p. 87-90 (1958), hereby incorporated by refer- 
ence as if fully set forth herein. Typically, such informa- 
tion consists of a distance attribute 210, the next hop 
attribute from the router 220, and perhaps a parent at- 
tribute for each node 230, as shown in FIG. 2, At each 
hop, during the executbn of the SPT algorithm in router 
S, the distance attribute 210 of destination node X rep- 
resents the distance of the shortest path found so far 
from the router S to node X, while the next hop attribute 
and the parent attribute respectively Identify the first hop 
from router S and the last hop to router S, 
[0020] The'data structure of FIG. 1 , however, contains 
more state information about each node in the network, 
without incurring significant computational complexity. 
More specifically, a given router S will maintain for each 
destination node X the shortest distance d(S, X) from 
the rout r S to X. This information can b computed with 
existing SPT algorithms such as those taught in Dijkstra 
and Bellman, supra. In addition, the distance, or cost w 
(S,p) for each outgoing link connecting the router S to a 
potential next hop p is also maintained. Furthermore, the 
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data structure keeps track of the length of the shortest 
path that usespasth next hop from S, which is denoted 
by dp(S. X). When the MPA execution terminates. dp(S. 
X) should equal the sum of w(S,P) and the shortest dis- 
tance found. d(p, X) from p to X. By maintaining this data 
structure, the MPA identifies viable next hops for desti- 
nation X using the principles discussed above: namely 
a viable path using next hop p exists if dp(S, X) - w(S, 
p)<d(S. X). 

[0021] While all next hops satisfying the above ine- 
quality are viable and can be revealed by the MPA, it is 
possible for additional viable next hops to exist which 
will not be revealed. By searching through paths, the da- 
ta structure at every node can be updated, increasing 
the number of viable next hops. As long as the shortest 
distance to every node Is computed correctly, the result- 
ing viable next hops will not lead to a loop. 
[0022] With the alternative viable paths for a given 
destination, alternate paths can be constructed by com- 
paring the MPA data structures associated with different 
destinations. Most SPT algorithms known in the art con- 
stantly compare the distance attributes of two neighbor- 
ing nodes. The distance attribute of a node eventually 
converges to the shortest distance from the source to 
that node. The distance attribute of node X from source 
S computed by the SPT algorithm is contained in the 
corresponding field 210 of the shortest distance from S 
to X and is updated by the SPT algorithm, independently 
of other operations of the MPA. 

[0023] Referring to FIG. 3, the extended routing algo- 
rithm 300 in accordance with the present invention com- 
prises these modules: MPA Algorithm 301 , Convention- 
al Routing Algorithm 302 and Routing Protocol Memory 
303. The MPA and Conventional Algorithms 301 and 
302, respectively, while possible to be implemented with 
hardware, typically comprise software executed on a 
standard microprocessor having the computing power 
of at least a Power PC or Pentium grade chip. Routing 
Protocol Memory 303 typically comprises at least 4 
megabytes ("MB") of RAM, with 3MB typically allocated 
to the Conventional Algorithm 302 and 1MB allocated 
to the MPA Algorithm. Furthermore, it is possible to pro- 
vide physically separate RAM chips for use by each of 
the algorithms 301 and 302. Forwarding Table 350 also 
typically comprises at least 4MB of RAM. 
[0024] Every time the SPT algorithm, represented by 
block 302, makes a comparison between nodes a and 
b, the distance attribute from source S to b, d^jp^(b), is 
equal to the smaller values between its old value and 
the sum of d^jp|(a) +w(a, b). 

[0025] Furthermore, with each comparison by the 
SPT algorithm 302 between nodes a and b, MPA block 
301 Is triggered and starts Its own comparisons. For 
ev ry next hop p from S, the distance from S to b is set 
to the minimum of its old value and dp(S, a) +w(a, b). 
With the termination of the SPT algorithm, the MPA also 
terminates and the data structure is updat d in routing 
protocol memory 303. 



[0026] With the availability of viable next hops the 
router can select the actual next hop for forwarding a 
packet based on a variety of criteria such as round robin , 
network load, or shortest distance. Those skilled in the 
5 art will be able to devise various selection criteria. 
[0027] Indeed, one such criteria can be used to deter- 
mine satisfactory approximate solutions to QoS routing 
by decoupling the underiying optimization problem dis- 
cussed above, into local and global computations. First, 
10 multiple viable paths that satisfy the global or less fre- 
quently changed QoS constraints are computed. Then 
among these viable paths, one that is optimized with re- 
spect to the local or more frequently changed QoS pa- 
rameters can be determined. 
IS [0028] With respect to load balancing it should be not- 
ed that implementation of such a scheme may require 
a means to preserve the FIFO arrival of packets where 
FIFO is assumed by the upper layer protocol at the des- 
tination router. It may also be necessary to provide ad- 
ditional buffer means and processing capabilities at the 
destination router, especially where FIFO is not strictly 
adhered to. 

[0029] In addition each router can compute and store 
in a forwarding table such as 350 in FIG. 3, each viable 
next hop for a given destination or maintain only a single 
viable next hop in forwarding table 350 while maintain- 
ing additional next hop infomnation in the routing proto- 
col memory 303. 

[0030] By maintaining multiple viable next hops in a 
router for each destination, recovery from link failure is 
much faster than heretofore possible, since the recovery 
mechanism using alternate paths can be implemented 
locally by the router, and there is almost no delay in- 
curred by the link failure. On the other hand, without 
MPA, after detecting a link failure, the router would need 
to broadcast throughout the network the link failure, then 
recompute the shortest path tree to determine a new al- 
ternative path. 

[0031] A more exhaustive search of alternate next 
hops can be obtained by running a set of independent 
SPT algorithms in parallel. For every next hop Pj any 
SPT algorithm including a dynamic algorithm, i.e. , an al- 
gorithm which can compute a new SPT by readjusting 
the old SPT, is executed using only the distance attribute 
dp(S,X) in the data structure associated with the desti- 
nation X. (While a static algorithm recomputes an entire 
shortest path, a dynamic algorithm recognizes that the 
majority of the path segments will remain the same and 
therefore only certain segments are recomputed). After 
the SPT algorithm is executed, the data structure is up- 
dated and all of the next hops pj that satisfy the above 
inequality are obtained. 



1. A method for routing a data packet from a router to 
a destination node in a packet switching network. 
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said data packet originally from a source node in 
said packet switching network, said method com- 
prising the steps of: 

determining one or more viable next hops from 
said router, each of said viable next hops lying 
on a path from said source node to said desti- 
nation node, said path having a distance which 
decreases at each next hop along said path un- 
til said destination node; 
storing in a first memory means at said router 
said one or more viable next hops; and 
selecting a first one of said one or more viable 
next hops to forward said data packet. 

2. A method according to Claim 1 wherein each of said 
viable next hops are in a path from said destination 
whose shortest distance is less than the absolute 
shortest distance from said source node to said 
destination node. 

3. A method according to Claim 1 further comprising 
the step of selecting a second one of said one or 
more viable next hops if it is determined that a link 
failure exists along said path of said first viable next 
hops and a second viable next hop is stored in said 
first memory means. 

4. A method according to Claim 1 wherein each of said 
viable next hops leads to a loop free path from said 
source node to said destination node, but is not con- 
strained to lead to a shortest path between said 
source node and said destination node. 

5. A method according to Claim 1 wherein said select- 
ing steps are constrained to select a viable next hop 
leading to the shortest path as compared with all of 
said one or more viable next hops stored in said first 
memory means. 

6. A method for determining multiple loop free paths 
from a source node to a destination node in a packet 
switching network, said packet switching network 
comprising a plurality of routers, said method com- 
prising the step of selecting at each of said routers, 
one or more viable next hops for forwarding a data 
packet originating from a source node, to a destina- 
tion mode, each of said viable next hops con- 
strained by the inequality that the distance from said 
next hop to said destination node is less than the 
shortest distance from said source node to said 
destination node. 

7. A router for use in a packet switching network for 
routing a data packet originating from a source 
node, to a destination node, comprising: 

a data structure comprising, 



the shortest distance from said router to a des- 
tination node; 

the distance to each next hop from said router; 
and 

5 for each of said next hops, the distance of the 

shortest path from said router to said destina- 
tion node using said next hop; 
computer. implemented means for determining 
one or more viable next hops from said data 

10 structure, each of said viable next hops satis-. 

tying the inequality that the difference between 
the distance from said router to said destination 
node traversing one of said viable next hops, 
and the cost from said router to said next hop 

15 is less than the shortest distance from said rout- 

er to said destination node. 

8. A data structure tor use in a router in a packet 
switching network, comprising: 

20 

the shortest distance from said router to a des- 
tination node; 

the cost of each next hop from said router; and 
for each of said next hops, the distance of the 
2S shortest path from said router to said destina- 

tion node using said next hop. 
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even where less than all of the routers use the data 
structure and multiple path scheme described herein. 
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